The Last-Seen Image: an Image-Based Approach for Finding Lost Objects Using a Head-Mounted Display

Considering the current development of commercial head-mounted displays (HMD), it is likely that HMDs will be widely used in the near future. Therefore, it becomes feasible to build systems that rely on HMD technology. Real-world search engines aim at supporting the user’s search capabilities in the real world. HMDs are a possible device to guide a user towards the location of an object. To ensure a high usability, it is essential to find suitable location representations of search results on an HMD. Based on previous findings, we present a novel location representation called “last-seen image” to locate objects in a known environment (e.g. the user’s home or office building). The last-seen image shows the picture of a sought object including the surrounding context of the object. We implemented a prototype on an HMD using WiFi indoor positioning to provide the proposed visualization as well as a map visualization. We conducted a user study comparing our proposed last-seen image approach to a map based approach. The last-seen image showed to be significantly faster for finding harder hidden objects compared to the map representation. However, the map was favored for finding the correct room. Therefore, we propose a hybrid system using the map representation to find the correct room and using the last-seen image to find the object on room-level.